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University  of  Maryland,  College  Park 

1-  Main  results 

The  following  is  an  abridged  list  of  the  main  achievements  that  resulted  from  research 
supported  by  this  grant: 

•  In  collaboration  with  Dr.  V.  Gupta  (UND),  we  were  the  first  to  characterize  the 
necessary  and  sufficient  conditions  for  stabilization  in  a  networked  control 
framework  where  there  is  a  channel  between  the  controller  and  the  actuator. 

•  Under  my  supervision,  Mr.  Kenneth  Running  has  solved  the  problem  of  optimal 
tracking  for  Markovian  Jump  Linear  Systems.  This  can  be  applied  to  networked 
control  systems  whose  interconnecting  links  feature  random  erasures. 

•  In  collaboration  with  Dr.  J.  Goncalves  (U.  Cambridge,  UK),  we  have  developed  a 
new  method  to  design  and  analyze  master  equations,  which  are  a  mainstay  model 
for  large-scale  systems  and  biological  networks. 

•  In  collaboration  with  Drs.  V.  Gupta  and  John  Baras  (UMD),  we  have  solved  the 
problem  of  optimal  sensor  allocation  for  control  in  the  presence  of  noise  and 
communication  constraints. 

•  In  collaboration  with  Tsachy  Weissman  (Stanford  U.),  we  have  developed  a 
framework  and  new  performance  analysis  methodology  for  control-inspired 
communication  schemes  for  Gaussian  channels  in  the  presence  of  quantized 
feedback. 

•  Under  my  supervision,  Mr.  Gabriel  Lipsa  identified  a  new  class  of  team  decision 
problems  that  extends  Witsenhausen’s  original  example.  Mr.  Lipsa  also  proposed 
and  analyzed  the  performance  of  new  methods  for  the  optimal  design  of  such  a 
new  class  of  systems. 

•  Under  my  supervision,  Mr.  Lipsa  solved  the  long  standing  problem  of  designing 
jointly  optimal  transmitters  and  receivers  for  distributed  state  estimation  in  the 
presence  of  communication  costs. 


2-  Journal  Publications 


G.  Lipsa  and  N.  C.  Martins,  “ Optimal  memoryless  control  in  Gaussian  noise:  A  simple 
counterexample Volume  47,  Issue  3,  March  2011,  Pages  552-558 
In  this  paper,  we  investigate  control  strategies  for  a  scalar,  one-step  delay  system  in 
discrete-time,  i.e.,  the  state  of  the  system  is  the  input  delayed  by  one  time  unit.  In  contrast 
with  classical  approaches,  here  the  control  action  must  be  a  memoryless  function  of  the 
output  of  the  plant,  which  comprises  the  current  state  corrupted  by  measurement  noise. 
We  adopt  a  first  order  state-space  representation  for  the  delay  system,  where  the  initial 
state  is  a  Gaussian  random  variable.  In  addition,  we  assume  that  the  measurement  noise  is 
drawn  from  a  white  and  Gaussian  process  with  zero  mean  and  constant  variance. 
Perfonnance  evaluation  is  carried  out  via  a  finite-time  quadratic  cost  that  combines  the 
second  moment  of  the  control  signal,  and  the  second  moment  of  the  difference  between 
the  initial  state  and  the  state  at  the  final  time.  We  show  that  if  the  time-horizon  is  one  or 
two  then  the  optimal  control  is  a  linear  function  of  the  plant’s  output,  while  for  a 
sufficiently  large  horizon  a  control  taking  on  only  two  values  will  outperform  the  optimal 
affine  solution.  This  paper  complements  the  well-known  counterexample  by  Hans 
Witsenhausen,  which  showed  that  the  solution  to  a  linear,  quadratic  and  Gaussian  optimal 
control  paradigm  might  be  nonlinear.  Witsenhausen’ s  counterexample  considered  an 
optimization  horizon  with  two  time-steps  (two  stage  control).  In  contrast  with 
Witsenhausen’ s  work,  the  solution  to  our  counterexample  is  linear  for  one  and  two  stages 
but  it  becomes  nonlinear  as  the  number  of  stages  is  increased.  The  fact  that  our  paradigm 
leads  to  nonlinear  solutions,  in  the  multi-stage  case,  could  not  be  predicted  from  prior 
results.  In  contrast  to  prior  work,  the  validity  of  our  counterexample  is  based  on 
analytical  proof  methods.  Our  proof  technique  rests  on  a  simple  nonlinear  strategy  that  is 
useful  in  its  own  right,  since  it  outperforms  any  affine  solution. 

G.  Lipsa  and  N.  C.  Martins,  “ Remote  State  Estimation  With  Communication  Costs  for 
First-Order  LTI  Systems,”  IEEE  Transactions  on  Automatic  Control,  Vol.  56,  No.  9, 
September  2011,  Pages  2013-2025 

Consider  a  first  order  linear  time-invariant  discrete  time  system  driven  by  process  noise,  a 
pre-processor  that  accepts  causal  measurements  of  the  state  of  the  system,  and  a  state 
esti-  mator.  The  pre-processor  and  the  state  estimator  are  not  co-lo-  cated,  and,  at  every 
time-step,  the  pre-processor  transmits  either  a  real  number  or  a  free  symbol  to  the 
estimator.  We  seek  the  pre-pro-  cessor  and  the  estimator  that  jointly  minimize  a  cost  that 
com-  bines  two  terms;  the  expected  squared  state  estimation  error  and  a  communication 
cost.  In  our  formulation,  the  transmission  of  a  real  number  from  the  pre-processor  to  the 
estimator  incurs  a  positive  cost  while  free  symbols  induce  zero  cost.  This  paper  proves 
analyt-  ically  that  a  symmetric  threshold  policy  at  the  pre-processor  and  a  Kalman-like 
filter  at  the  estimator,  which  updates  its  estimate  lin-  early  in  the  presence  of  free 
symbols,  are  jointly  optimal  for  our  problem. 


V  Gupta  and  N.  C.  Martins,  “ On  Stability  in  the  Presence  of  Analog  Erasure  Channel 
Between  the  Controller  and  the  Actuator,”  IEEE  Transactions  on  Automatic  Control, 
Vol.  55,  No.  1,  January  2010,  Pages  1 75-1 79 

Consider  a  discrete-time  networked  control  system,  in  which  the  controller  has  direct 
access  to  noisy  measurements  of  the  output  of  the  plant.  However,  information  flows 
from  the  controller  to  the  actuator  via  a  channel  that  features  Bernoulli  erasure  events.  If 
an  erasure  occurs,  the  channel  outputs  an  erasure  symbol;  otherwise,  it  transmits  a  real 
finite-dimensional  vector.  We  determine  necessary  and  sufficient  conditions  for  the 
stabilizability  of  an  unstable  linear  time-invariant  finite-dimensional  plant.  Given  a 
minimal  state-space  representation  for  the  plant,  the  necessary  and  sufficient  conditions 
for  stabilizability  are  expressed  in  terms  of  the  probability  of  erasure  at  the  channel  and 
of  the  spectral  radius  of  the  one-step  state  transition  matrix.  There  are  two  main  results  in 
the  technical  note.  The  first  result  shows  that  if  the  actuator  has  processing  capabilities, 
then  the  necessary  and  sufficient  conditions  for  stabilizability  remain  unchanged  with  or 
without  acknowledgements  from  the  actuator  to  the  controller.  The  second  result  shows 
that  the  stabilizability  conditions  are  identical  for  two  types  of  actuators:  (Type  I) 
Processing  at  the  actuator  has  access  to  the  plant's  model;  (Type  II)  Processing  at  the 
actuator  uses  a  universal  algorithm  that  does  not  depend  on  the  model  of  the  plant.  Thus, 
neither  the  knowledge  of  the  model  of  the  plant  at  the  actuator,  nor  the  presence  of 
acknowledgements  from  the  actuator  to  the  controller,  can  be  used  to  alter  or  relax  the 
necessary  and  sufficient  conditions  for  stabilizability. 

V  Gupta  and  N.  C.  Martins,  “Optimal  tracking  control  across  erasure  communication 
links,  in  the  presence  of preview,”  Issue  16,  Vol  19,  Control  under  limited  information: 
Special  issue  (Part  I),  November  2009,  Pages:  1837-1850 

Consider  two  identical  discrete-time,  finite-dimensional,  linear  and  time-invariant 
systems  denoted  as  leader  and  follower.  The  leader  system  is  driven  by  a  deterministic 
control  input  and  by  a  zero-mean  white  Gaussian  process  noise.  In  this  paper,  we  address 
the  problem  of  designing  a  networked  control  scheme  for  the  follower  system  that 
guarantees  that  the  state  of  the  follower  system  tracks  the  state  of  the  leader  system 
optimally,  according  to  a  mean  squared  cost.  We  adopt  a  networked  control  scheme 
featuring  two  erasure  links  and  three  design  blocks:  a  controller  acting  on  the  follower 
system  and  two  encoders,  one  at  the  output  of  each  system.  The  controller  has  remote 
access  to  noisy  measurements  of  the  state  of  both  systems,  via  two  erasure  links  that  are 
used  to  connect  each  encoder  to  the  controller.  We  consider  erasure  links  whose  erasure 
events  occur  according  to  a  Bernoulli  process.  If  an  erasure  occurs  then  no  information  is 
transmitted,  otherwise  a  finite  vector  of  real  numbers  is  conveyed  through  the  link.  The 
purpose  of  the  encoders  is  to  process  noisy  measurements  of  the  output  of  each  system 
prior  to  transmission  over  the  corresponding  erasure  link.  While  the  encoder  of  the 
follower  system  has  access  to  current  measurements,  the  encoder  of  the  leader  system  has 
access  to  measurements  that  are  advanced  in  time  by  a  finite  time  interval  (also  denoted 
as  preview).  This  paper  describes  a  methodology  for  the  design  of  controller  and 
encoders  that  are  jointly  optimal,  with  respect  to  optimal  tracking  of  the  leader  system  by 
the  follower  system.  We  also  obtain  explicit  necessary  and  sufficient  conditions  for  the 
existence  of  a  scheme  that  guarantees  that  the  tracking  error  has  finite  second  moment. 


N.  C.  Martins  and  J.  M.  Goncalves,  “ A  Linear  Programming  Approach  to  Parameter 
Fitting  for  the  Master  Equation ,”  IEEE  Transactions  on  Automatic  Control,  Vol  54, 
Issue  10,  October,  2009,  Pages:  2451  -  2455 

This  technical  note  proposes  a  new  framework  for  the  design  of  continuous  time,  finite 
state  space  Markov  processes.  In  particular,  we  propose  a  paradigm  for  selecting  an 
optimal  matrix  within  a  p re- specified  pencil  of  transition  rate  matrices.  Given  any 
transition  rate  matrix  specifying  the  time-evolution  of  the  Markov  process,  we  propose  a 
class  of  figures  of  merit  that  upper-bounds  the  long-tenn  evolution  of  any  statistical 
moment.  We  show  that  optimization  with  respect  to  the  aforementioned  class  of  cost 
functions  is  tractable  via  dualization  and  linear  programming  methods.  In  addition,  we 
suggest  how  this  approach  can  be  used  as  a  tool  for  the  sub-optimal  design  of  the  master 
equation,  with  performance  guarantees.  Our  results  are  applied  to  illustrative  examples. 

K.  D.  Running  and  N.  C.  Martins,  “Optimal  Preview  Control  of  Markovian  Jump 
Linear  Systems,” IEEE  Transactions  on  Automatic  Control,  Volume  54,  Issue  9,  Sept. 
2009  Page(s):  2260  -  2266 

In  this  technical  note,  we  investigate  the  design  of  controllers,  for  discrete -time 
Markovian  jump  linear  systems,  that  achieve  optimal  reference  tracking  in  the  presence 
of  preview  (reference  look-ahead).  For  a  quadratic  cost  and  given  a  reference  sequence, 
we  obtain  the  optimal  solution  for  the  full  infonnation  case.  The  optimal  control  policy 
consists  of  the  additive  contribution  of  two  tenns:  a  feedforward  tenn  and  a  feedback 
term.  We  show  that  the  feedback  term  is  identical  to  the  standard  optimal  linear  quadratic 
regulator  for  Markovian  jump  linear  systems.  We  provide  explicit  formulas  for 
computing  the  feedforward  tenn,  including  an  analysis  of  convergence. 

Vijay  Gupta,  Nuno  C.  Martins  and  John  S.  Baras,  “Optimal  Output  Feedback  Control 
Using  Two  Remote  Sensors  Over  Erasure  Channels,”  IEEE  Transactions  on 
Automatic  Control,  Volume  54,  Issue7,  July  2009  Page(s):  1463  -  1476 
Consider  a  discrete-time,  linear  time-invariant  process,  two  sensors  and  one  controller. 
The  process  state  is  observed  in  the  presence  of  noise  by  the  sensors,  which  are  connected 
to  the  controller  via  links  that  feature  erasure.  If  a  link  transmits  successfully  then  a 
finite-dimensional  vector  of  real  numbers  is  conveyed  from  the  sensor  to  the  controller.  If 
an  erasure  event  occurs,  then  any  information  conveyed  over  the  link  is  lost.  This  paper 
addresses  the  problem  of  designing  the  maps  that  specify  the  processing  at  the  controller 
and  at  the  sensors  to  minimize  a  quadratic  cost  function.  When  the  information  is  lost 
over  the  links  either  in  an  independent  and  identically  distributed  (i.i.d.)  or  in  a  (time- 
homogeneous)  Markovian  fashion,  we  derive  necessary  and  sufficient  conditions  for  the 
existence  of  maps  such  that  the  process  is  stabilized  in  the  bounded  second  moment 
sense.  We  also  solve  the  optimal  design  problem  in  the  presence  of  delayed  noiseless 
acknowledgment  signals  at  the  sensors  from  the  controller  for  an  arbitrary  packet  drop 
pattern.  We  provide  explicit  recursive  schemes  to  implement  our  solution.  We  also 
indicate  how  our  approach  can  be  extended  to  situations  when  more  than  two  sensors  are 
available  and  when  the  sensors  can  cooperate.  The  analysis  also  carries  over  to  the  case 
when  each  point-to-point  erasure  link  connecting  the  sensors  and  the  controller  is 
replaced  by  a  network  of  erasure  links. 


Nuno  C.  Martins  and  Tsachy  Weissman,  “ Coding  for  Additive  White  Noise  Channels 
With  Feedback  Corrupted  by  Quantization  or  Bounded  Noise,  ”  IEEE  Transactions  on 
Information  Theory,  Volume  54,  Issue  9,  Sept.  2008  Page(s):  4274  -  4282 
We  present  coding  strategies,  which  are  variants  of  the  Schalkwijk-Kailath  scheme,  for 
communicating  reliably  over  additive  white  noise  channels  in  the  presence  of  corrupted 
feedback.  Our  framework  comprises  an  additive  white  forward  channel  and  a  feedback 
link.  We  consider  two  types  of  corruption  mechanisms  in  the  feedback  link.  The  first  is 
quantization  noise,  i.e.,  the  encoder  receives  the  quantized  values  of  the  past  outputs  of 
the  forward  channel.  The  quantization  is  uniform,  memoryless  and  time  invariant.  The 
second  corruption  mechanism  is  an  arbitrarily  distributed  additive  bounded  noise.  Here 
we  allow  symbol-by-symbol  encoding  at  the  input  to  the  feedback  link.  We  propose 
explicit  schemes  featuring  positive  infonnation  rate  and  positive  error  exponent.  If  the 
forward  channel  is  additive  white  Gaussian  (AWGN)  then,  as  the  amplitude  of  the  noise 
at  the  feedback  link  decreases  to  zero,  the  rate  of  our  schemes  converges  to  the  capacity 
of  the  channel.  Moreover,  the  probability  of  error  is  shown  to  converge  to  zero  at  a 
doubly  exponential  rate.  If  the  forward  channel  is  AWGN  and  the  feedback  link  consists 
of  an  additive  bounded  noise  channel,  with  signal-to-noise  ratio  (SNR)  constrained 
symbol-by-symbol  encoding,  then  our  schemes  achieve  rates  arbitrarily  close  to  capacity, 
in  the  limit  of  high  SNR  (at  the  feedback  link). 
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5-  Potential  Impact  on  DoD  Capabilities 

The  research  developed  in  this  grant  lead  to  a  significant  body  of  interrelated 
results  that  are  relevant  for  the  design  and  analysis  of  performance  of 
networked  control  systems.  These  results  may  have  an  impact  on  the  design 
and  provable  performance  certification  of  networks  of  semi-autonomous 
systems  that  are  required  to  execute  safety  critical  missions  with  high 
precision  in  the  presence  of  communication  constraints. 


